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ABSTRACT 

In fall 1993. Coffeyville Community College (CCC) in 
Kansas announced the implementation of a new program known as 
Quantifying Learning Outcomes (QLO) , one component of the college's 
Student Outcomes Assessment Plan. QLO calls for a clear articulation 
of CCC's goals regarding instructional support materials within the 
next 6 to 12 months; the revision of all course syllabi tc include 
clearly stated instructional competencies and outcomes; the use of 
pre- and post-tests in every class using test items indexed back tc 
specific learning outcomes; and on-going investigations of both 
student achievement and the level of appropriateness of course 
competencies and outcomes. To accomplish these objectives, in-house 
training and resource materials were made available to all faculty. 
In addition, PC-based TRACKER software was developec* to assist 
instructors in developing, categorizing, and analyzing instructional 
competencies and outcomes. TRACKER provides an easy method of 
maintaining student records concerning what outcomes have been 
achieved. TRACKER ' s investigations package enables instructors to run 
simple numerical and statistical tests on classroom data. The 
document includes a discussion of the terminology used in current 
instructional paradigms; a brochure explaining the QLO program; and 
an instructional booklet for faculty on the TRACKER system. (KP) 
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Quantifying Learning Outcomes: 
A Gentle Approach 



"Things are changing so fast that e'en the future isn't what it used to 
be." Yogi Berra 



In the Fall of 1993 Coffeyville Community College announced the implementation of a 
new program. This program calls for the revision of all course syllabi, the writing of 
instructional competencies and outcomes, and the use of pre and post tests in every class. 
This series of activities came to be referred to as "Q.L.O." or Quantifying Learning Outcomes. 

The factors which led to the decision to embrace these instructional strategies are 
many and include the Kansas State Department of Vocational Education's requirement to 
validate learning in all vocational programs. In addition, the North Central Association of 
Colleges and Schools decision to require all colleges and universities to document academic 
achievement in all course work especially those courses identified as core courses in the 
General Education sequence, played a decisive role in the launching of this program. Lastly, 
Coffeyville Community College has a seventy year history as a provider of quality higher 
education. The college sensed a "shift in the instructional paradigm" sweeping the nation and 
was quick to respond to this. 

The definition (McDaniel, 1994) of a "paradigm shift" suggests that it represents a set 
of rules and regulations (Written or unwritten) that does two things: 

1. Establishes or defines boundaries. 

2. It tells you how to behave within the boundaries to be successful. 

Spady & Marshall (McDaniel, 1994), leading proponents of Outcomes Based Education 
(O.B.E.), agree that " ... more, longer, and harder" must give way to " ... different, smarter, and 
better." 

McDaniel goes ahead to speculate that college professors in the future will increasingly 
be asked to: 

1. Define goals of instruction as measurable outcomes. 

2. Concentrate on the kinds of outcomes that promote critical thinking, problem 
solving, and creativity that will serve the future citizen in a changing world. 

3- Construct student evaluation opportunities that call for an integrating response. 



4. Adjust time spent by students so that mastery is the goal. 



5. Work to assure student success by using criterion-based evaluation rather than 
norm-based evaluation. 

6. Redesign curriculum around priority outcomes to be performed by students in 
a performance context. 

Anyone who has been in education for a while realizes that these "shifts in paradigms" 
occur periodically and in many instances cause significant upheaval among educators as they 
scramble to bring their efforts in line with the new paradigm. Interestingly, the pressures for 
change are usually exerted by individuals and groups from outside the educational system. 
Among the pressures brought to bear are: 

X. Excellence - To compete in the world market place, we must address the issue 
of higher level thinking and sophisticated technical skills. 

2. Accountability - Must prove that we are worth the increasing cost to the tax 
payer. 

3. Information - Print information as the basis for the Knowledge industry will no 
longer suffice. Emphasis must be placed on incorporating computer 
networking, databases, and the hypermedia technology. 

Common to all instructional paradigms is the vocabulary used to identify its key points. 
Currently the terms most in vogue include: 

1- Critical Thinking - Based on Bloom's taxonomy of cognitive skills, critical 
thinking would be typified by components of analysis, synthesis, and evaluation 
These skills involve the processing of information to arrive at generalities from 
which further information may be derived. (Inductive reasoning). 

A storm of controversy rages over the issue of rather or not "critical thinking" 
can be taught. That is, is it possible to teach critical thinking skills apart from 
subject matter. As McPeck(McPeck, 1990) puts it, " thinking is also about 
something, to think about nothing is a conceptual impossibility." Critical 
thinking guru, Richard Paul, believes that seminars on developing critical 
thinking skills apart from subject matter can be successful. 

2 - Understanding - Understanding represents an area of intense interest of recent 
years. Teachers quite often realize that students have acquired information but 
do not "understand" related concepts. According to the Harvard Graduate 
School of Education in their recent "Teaching for Understanding Project", 
(Perkins & Blythe, 1994) the following definition for "understanding" is 
proposed: 

" ... The ability to do a number of thought demanding activities such as: 
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explaining, finding evidence, applying, analogizing, and representing the topic 
in a new way (Understanding Performance)." Perkins and Blythe in their article 
"Putting Understanding Up Front" suggest that to teach a youngster to roller 
skate, rather than having the child read books about it, or watch someone else 
do it, the child must be engaged in that particular activity himself. The 
"mainstay" of learning for understanding is the engagement on the pan of the 
student of those performances which involve higher levels of learning. 

The Harvard Project proposes a four part framework for the teaching of 
understanding: 

A. Generative Topics 

Not all topics lend themselves as well for the teaching of 
understanding. Statistics does well in this regard however, the quadratic 
formula, while equally important, does not. 

B. Understanding Goals 

Too many generative topics can occur. Some focus must be 
used. "Students will understand the features that make the Boston Tea 
Party like other political protests from historical various historical 
periods." 

C. Understanding Performances 

Must extend over a unit of study, several weeks to a month. 
Preferably students will end up doing some culminating activity such as 
an extended essay or exhibition. 

D. Ongoing Assessment 

Students need continual feedback from the instructor or peers. 
They need opportunity for reflection on "public criteria", or outside 
scrutiny of their project. 

Metacoenition 

Metacognition is the newest of the "hot" topics to consider while 
designing instructional systems. According to WeinertfWeinert, 1987), 
"Metacognition are second order cognitions; thoughts about thoughts, 
knowledge about knowledge, reflections about actions." Requiring students to 
consider "how" they learn best, what works for them, what study habits lead to 
success, etc. In addition to these, suggest that the students consider concepts 
in relation to other concepts, what they have in common, how best they might 
learn new concepts based on material already learned. 
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4. Competencies/Outcomes 



A search of the literature quickly reveals no overall agreement on the 
definition of the terms "competencies" and "outcomes". To compound this 
problem, teachers who have been around for awhile may find some confusion 
with these terms and earlier terms such as behavioral objectives and goals. 

Initially one is tempted to suggest that which terms a person use makes 
little difference as long as the instructional process is served. However, given 
the reality of the times, it is advantageous for all us to use the terminology 
suggested by the various stare departments of education and regional 
accrediting agencies. This is especially true if we are seeking funding or some 
type of program approval from external authorities. 

One definition for competencies according to McAshanfMcAShan, 1979), 
is " ... the knowledge, skills, and abilities or capabilities that a person achieves, 
which becomes pan of his or her being to the extent that he or she can 
satisfactorily perform particular cognitive, affective, and psychomotor behaviors. 
They represent the intents of a program and are stated as specific goals to be 
achieved." 

Competencies at Coffeyville Community College are viewed as 
statements of learning supported by observable and measurable "outcomes". 
This is not to say that others are wrong if they use these terms differently, but 
that there should be a set of agreed upon terms in use by everyone 
participating in the same instructional activities. 



Direct Assessment of Student Academic Achievement 

At 

Coffeyville Community College 



Coffeyville Community College has implemented a bold and ambitious program 
of instructional assessment. This program is one component of the Student Outcomes 
Assessment Plan developed and adopted by the college in the Fall of 1993. The 
identified benchmarks of this program are: 

1. Identification of Courses 

Courses designated to be brought into the mainstream of activities, 

which by this time will be essentially all academic and vocational offerings. 



Identification of Specific Performance Objectives 

(Competencies). Competencies and outcomes have been written for 
essentially all course. 



3- Evaluation/Revision of Current Svllabi 

Syllabi revision for all classes taught. 



4. Pre/Post Testing 

Pre and post testing for each class offered. 



5- Correlation of Exams and Specific Performance Objectives 

This correlation activity will provide for close monitoring of instructional 
efficacy. 



6. Systematic Evaluation of Instruction 

A systematic means of analyzing student achievement feedback allowing 
for the suggested modifications of instructional materials and/or procedures. 



TRACKER SYSTEM SOFTWARE 

Any attempt to implement a comprehensive, quantifiable system of instructional 
evaluation involves a number of reports to be generated, files to be kept, and no little amount 
of math to be used in the documentation process. Taking advantage of the availability of pe s 
by the faculty both in their offices and at home, a complete software system was developed to 
assist in "tracking" student achievement, writing competencies, and quantifying learning 
outcomes. 

The faculty of Coffeyville Community College was invited to learn this software by 
means of several short training sessions. Although no coercion was used, approximately one- 
third of the faculty took advantage of the opportunity to attend training sessions, with the 
understanding that upon completing the three sessions, they would be given a free copy of 
this pc-based software for use op their own computer. 

A partial list of reports generated by "TRACKER" includes: 

A. Analysis & Categorization of Competencies 
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B. 



Summary of Class Academic Achievement 



C. 



Individual Academic Achievement 



D. 



Pre/Post Test Analysis 



E. 



Correlation of Items 



F. 



Looking for Predictors 



Conclusion 



Coffeyville Community College is excited about the changes occurring within its 
instructional areas. Much has been learned, and there is much yet to learn about the process 
of quantifying student academic achievement. While results are sought after, it is the process 
that is most important. Anytime a group of professionals join together to seek "improvement" 
of instruction, good things happen and all benefit. 



************************************ 
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